Skip to content

Comments

Improvements to the Draft Server Auth Conformance tests#155

Open
wdawson wants to merge 37 commits intomodelcontextprotocol:server-auth-conformancefrom
ArcadeAI:wils/server-auth-conformance
Open

Improvements to the Draft Server Auth Conformance tests#155
wdawson wants to merge 37 commits intomodelcontextprotocol:server-auth-conformancefrom
ArcadeAI:wils/server-auth-conformance

Conversation

@wdawson
Copy link

@wdawson wdawson commented Feb 19, 2026

Updates server auth conformance tests with various capabilities and merges main

Motivation and Context

I noticed that there were several gaps in the existing draft PR #105 and wanted to fill some of them in. This adds:

  • Validation for the PRM resource field
  • Adds failures and warnings for bad WWW-Authenticate header behavior
  • Adds a check for how the server responds to invalid tokens
  • Streamlined and unified client registration checks:
    • Added CIMD as the preferred option but tests DCR standalone if server claims support
    • CIMD is supported by adding a file and using GitHub to fetch it. Other options can be considered too, like something on modelcontextprotocol.io. I tested locally by overriding it and serving with a tunnel, which could be preffered, but the GitHub option will be zero config once this is merged.
    • Provides support (via CLI args) for pre-registered client support and tests CIMD and DCR when provided if server claims support
  • Fixes a lingering XSS issue in the callback server
  • Merges to main with various newer options (like outputDir, specVersion, etc.)

This DOES NOT yet address running other suites for servers that need auth. I was debating merging those test runners together, but that felt like a bigger lift than this is already.

How Has This Been Tested?

Ran this against an Arcade.dev Gateway server with all options:

Status Check Description
SUCCESS Invalid Token Rejected Server returns 401 for requests with invalid Bearer token
SUCCESS Unauthenticated Request Returns 401 Server returns 401 Unauthorized for unauthenticated MCP requests
SUCCESS WWW-Authenticate Header Present Server includes WWW-Authenticate header with Bearer scheme in 401 response
INFO Resource Metadata URL in WWW-Authenticate WWW-Authenticate header includes resource_metadata parameter
SUCCESS Scope in WWW-Authenticate Server includes scope parameter in WWW-Authenticate header
SUCCESS Protected Resource Metadata Discovery Client discovered Protected Resource Metadata endpoint
SUCCESS PRM Resource Field Protected Resource Metadata includes resource field matching server URL
SUCCESS PRM Contains Authorization Servers Protected Resource Metadata includes authorization_servers array
SUCCESS Authorization Server Metadata Discovery Client discovered Authorization Server metadata
SUCCESS AS Metadata Required Fields Authorization Server metadata includes all required fields
SUCCESS AS Supports CIMD Authorization server advertises client_id_metadata_document_supported
SUCCESS AS Supports DCR Authorization server advertises registration_endpoint
SUCCESS Token Acquisition Client obtained access token from token endpoint
SUCCESS Token Response Contains Access Token Token response includes access_token
SUCCESS Authenticated MCP Request Succeeds MCP request with Bearer token succeeds
SUCCESS OAuth Flow Completion Complete OAuth authentication flow succeeded
SUCCESS Dynamic Client Registration Server accepted Dynamic Client Registration
SUCCESS DCR Response Contains Client Credentials DCR response includes client_id
SUCCESS CIMD Authentication Flow AS accepts URL-based client_id via CIMD authentication flow

Summary: 18 SUCCESS, 0 FAILURE, 0 WARNING, 1 INFO

Breaking Changes

None that I'm aware of

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update

Checklist

  • I have read the MCP Documentation
  • My code follows the repository's style guidelines
  • New and existing tests pass locally
  • I have added appropriate error handling
  • I have added or updated documentation as needed

Additional context

This is meant to be merged into #105, and additional work may be needed to get the end result into a mergeable state.

pcarleton and others added 30 commits January 14, 2026 20:16
Add -o, --output-dir option to both client and server commands.
By default, no results directory is created - only console output.
When -o is specified, results are saved to that directory with
timestamped subdirectories.

This eliminates the need to add 'results/' to .gitignore in every
repo where conformance tests are run.

Claude-Generated-By: Claude Code (cli/claude-opus-4-5=100%)
Claude-Steers: 0
Claude-Permission-Prompts: 2
Claude-Escapes: 0
…odelcontextprotocol#113)

Add support for SDKs that don't yet pass all conformance tests to still
run conformance in CI without failing, by specifying a YAML baseline file
of known/expected failures.

CLI changes:
- Add --expected-failures <path> flag to both server and client commands
- Exit 0 if only expected failures occur
- Exit 1 if unexpected failures occur (regression)
- Exit 1 if an expected failure now passes (stale baseline, must update)

GitHub Action (action.yml):
- Composite action that builds from source and runs conformance tests
- SDKs reference it as: uses: modelcontextprotocol/conformance@<ref>
- Inputs: mode, url, command, expected-failures, suite, scenario, etc.

New files:
- src/expected-failures.ts: YAML loading, baseline evaluation, reporting
- src/expected-failures.test.ts: 16 unit tests
- action.yml: composite GitHub Action

Closes modelcontextprotocol#99
…tocol#115)

* feat: add DNS rebinding protection conformance tests

Add a new server conformance scenario to verify that localhost MCP servers
properly validate Host headers to prevent DNS rebinding attacks.

Checks:
- localhost-host-rebinding-rejected: Verifies server returns 403 for
  non-localhost Host headers (e.g., evil.example.com)
- localhost-host-valid-accepted: Verifies server accepts requests with
  valid localhost Host headers

Also updates the everything-server example to use createMcpExpressApp()
which includes DNS rebinding protection by default.

Closes modelcontextprotocol#103

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: add negative test case for DNS rebinding protection

Add a minimal MCP server that intentionally omits DNS rebinding protection
to serve as a negative test case. This server is expected to FAIL the
dns-rebinding-protection scenario.

Also update README to document the negative test case.

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: accept any 4xx status code for DNS rebinding rejection

Update the DNS rebinding protection test to accept any 4xx status code
as a valid rejection response, not just 403. This accommodates different
HTTP semantics:
- 403 Forbidden: Per MCP spec
- 421 Misdirected Request: RFC 7540 (semantically correct for DNS rebinding)
- Other 4xx: Implementation-specific error codes

* refactor: address PR review comments

- Simplify negative test server (remove event store, sessions)
- Update spec reference URL to security_best_practices
- Clarify scope: localhost servers without HTTPS/auth
- DRY up checks.push using spread operator
- Fix valid host check to require 2xx response

* fix: include error details in catch block

* refactor: simplify negative test server to stateless

* refactor: check both Host and Origin headers for DNS rebinding protection

- Send both Host and Origin headers in test requests so servers checking
  either header will pass the conformance test
- Add second spec reference for Origin check (transports#security-warning)
- Update descriptions to mention both Host and Origin headers
- Fix protocol version to 2025-11-25
- Remove unnecessary 'as const' type assertions

---------

Co-authored-by: Claude <noreply@anthropic.com>
…tocol#117)

* feat: add core and extensions suites for SDK tiering

Add new suites to support SDK tiering requirements:

Client suites:
- core (19 scenarios): Tier 1 requirements (base + auth)
- extensions (2 scenarios): Optional protocol extensions (client credentials)
- all (21 scenarios): Everything (core + extensions)
- auth (15 scenarios): Auth scenarios only (excludes extensions)

Server suites:
- core: Alias for 'active' (tier 1 requirements)

This allows SDK developers to run --suite core for tier 1 certification
while keeping extension tests available but not blocking.

* feat: promote elicitation tests from pending to active

Move 3 elicitation scenarios to active suite:
- elicitation-sep1330-enums
- tools-call-elicitation
- elicitation-sep1034-defaults

These tests are now passing in the Python SDK and ready for
SDK tiering requirements.

Remaining pending tests (2):
- json-schema-2020-12 (blocked on SDK PR #1135)
- server-sse-polling (blocked on SDK PR #1129)
* feat: add pre-registration conformance test

Add conformance test for OAuth pre-registration flow where the server
does NOT support Dynamic Client Registration (DCR) and clients must
use pre-configured static credentials.

This addresses issue modelcontextprotocol#34 (Client Registration Methods) - specifically
the pre-registration section that was previously not covered.

Changes:
- Add disableDynamicRegistration option to createAuthServer helper
- Add auth/pre-registration scenario with pre-registered client creds
- Add MCP_PREREGISTRATION spec reference
- Add context schema for pre-registration credentials
- Add withOAuthRetryWithProvider helper for pre-configured providers
- Add negative test client (auth-test-attempts-dcr.ts) that ignores
  pre-registered credentials and fails

The scenario verifies that when a server does not advertise
registration_endpoint in its OAuth metadata, compliant clients
use pre-registered credentials passed via context instead of
attempting DCR.

Closes modelcontextprotocol#34

* fix: add redirect_uris to saveClientInformation call

* chore: remove unused WithOAuthRetryOptions interface
…protocol#121)

* docs: add SDK integration guide and fix spec references

- Add SDK_INTEGRATION.md with guide for SDK maintainers on integrating conformance tests
- Link to guide from README.md
- Update spec references to use 2025-11-25 instead of draft URLs

* fix: prettier formatting and correct action version to v0.1.10
* feat: add PKCE conformance tests

Adds conformance checks for PKCE requirements from the MCP auth spec:

- pkce-code-challenge-sent: Verifies client sends code_challenge in auth request
- pkce-s256-method-used: Verifies client uses S256 code challenge method
- pkce-code-verifier-sent: Verifies client sends code_verifier in token request
- pkce-verifier-matches-challenge: Validates BASE64URL(SHA256(verifier)) = challenge

Also adds auth/pkce-no-s256-support scenario that tests clients correctly
refuse when S256 is not in code_challenge_methods_supported.

All PKCE checks are added to the createAuthServer helper, so they automatically
apply to all existing auth scenarios.

Fixes modelcontextprotocol#77

* feat: add bad client example that skips PKCE

Adds auth-test-no-pkce.ts which implements a custom OAuth flow that
deliberately skips PKCE (no code_challenge in auth request, no
code_verifier in token request).

Also adds negative test to verify the conformance suite correctly
detects this non-compliant behavior.

* chore: skip pkce-no-s256-support scenario for now

* chore: fix formatting

* refactor: remove pkce-no-s256-support scenario and always validate PKCE

- Remove pkce-no-s256-support scenario (was skipped anyway)
- Make pkce-verifier-matches-challenge fail if either code_challenge
  or code_verifier is missing, rather than only running when both present
- Update negative test to expect the new failure
…tprotocol#118)

* feat: add resource parameter validation tests (RFC 8707)

Adds conformance tests for OAuth Resource Indicators (RFC 8707) implementation:

1. Resource parameter checks added to token-endpoint-auth-basic scenario:
   - resource-parameter-in-authorization: Verify resource in auth request
   - resource-parameter-in-token: Verify resource in token request
   - resource-parameter-valid-uri: Verify valid canonical URI
   - resource-parameter-consistency: Verify consistency between requests

2. New auth/resource-mismatch scenario:
   - Tests that client rejects when PRM resource doesn't match server URL
   - Server returns mismatched resource in PRM
   - Test passes if client does NOT proceed with authorization

Also adds spec references for RFC 8707 and MCP resource parameter spec.

Closes modelcontextprotocol#33

* fix: make resource consistency check a FAILURE and remove dead code

- Change resource parameter consistency from WARNING to FAILURE
- Remove unreachable protocol check in validateCanonicalUri
  (URL constructor already validates scheme presence)
…es (modelcontextprotocol#123)

* Add allowClientError to Scenario interface for expected client failures

* Apply suggestion from @pcarleton

Co-authored-by: Paul Carleton <paulcarletonjr@gmail.com>

---------

Co-authored-by: Paul Carleton <paulcarletonjr@gmail.com>
…release testing (modelcontextprotocol#125)

- Remove NPM_TOKEN / NODE_AUTH_TOKEN — publish now uses OIDC trusted
  publishing (no secrets needed)
- Add workflow_dispatch trigger with prerelease checkbox: publishes to
  alpha dist-tag so we can verify OIDC works without touching latest
- Drop --provenance flag (included automatically with trusted publishing)

The release event path still publishes to latest as before; the only
change is the auth mechanism.

To test: bump version to x.y.z-alpha.0 on a branch, then run workflow
with prerelease checked.
…odelcontextprotocol#127)

Move auth/2025-03-26-oauth-metadata-backcompat and
auth/2025-03-26-oauth-endpoint-fallback out of the required
authScenariosList into a new backcompatScenariosList.

These test backward compatibility with the old 2025-03-26 auth spec
(no PRM, OAuth metadata at server root) which is not part of the
current spec requirements.

- Add new 'backcompat' client suite
- Back-compat scenarios remain in the 'all' suite
- Removed from 'core' and 'auth' suites
- Add separate test describe block for back-compat scenarios

Closes modelcontextprotocol#126
* chore: bump version to 0.1.12-alpha.0 for OIDC trusted publishing test

* ci: rename prerelease checkbox to publish_alpha, clarify no-publish default

* 0.1.12
Bumps [hono](https://github.com/honojs/hono) from 4.11.4 to 4.11.7.
- [Release notes](https://github.com/honojs/hono/releases)
- [Commits](honojs/hono@v4.11.4...v4.11.7)

---
updated-dependencies:
- dependency-name: hono
  dependency-version: 4.11.7
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
modelcontextprotocol#134)

* fix: bump SDK to 1.26.0, add session management to tools_call scenario

SDK 1.26.0 made Protocol.connect() throw if already connected to a
transport. The tools_call scenario was calling server.connect(transport)
on every request with the same Server instance, which now throws.

Fixed by adding proper session management: each initialize request
creates a new Server + Transport pair, and subsequent requests are
routed to the correct transport via the mcp-session-id header.

* fix: return 404 for invalid session IDs in tools_call POST handler

The else branch was returning 400 for both missing-session and
invalid/stale-session cases. The MCP spec requires 404 for invalid
session IDs and 400 only for non-initialization requests without
any session ID.

* refactor: keep tools_call stateless, create fresh server per request

Instead of adding session management, simply create a new Server
instance per request. This preserves the original stateless design
while fixing the SDK 1.26.0 Protocol.connect() restriction.
…xtprotocol#141)

The auth test server helper and no-dns-rebinding-protection example
reused a single Server instance across requests, calling connect()
on each new transport. SDK v1.26.0's security fix (GHSA-345p-7cg4-v4c7)
now throws "Already connected to a transport" in this case.

Fix by wrapping Server creation in a factory function called per
request, matching the pattern already used by tools_call.ts and
everything-server.ts.

Also updates no-dns-rebinding-protection.ts to use registerTool()
instead of the deprecated tool() API.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add tier-check CLI for SDK tier assessment

Adds a 'tier-check' subcommand to the conformance tool that automates
SDK tier assessment against SEP-1730 criteria.

Checks performed:
- Conformance test pass rate (via everything-server)
- GitHub label taxonomy (priority/status/area labels)
- Issue triage SLA compliance
- P0 bug resolution tracking
- Stable release detection
- Required file existence (CHANGELOG, SECURITY, etc.)
- Spec tracking (SDK release within 30d of spec release)

Also includes a Claude Code skill (skills/mcp-sdk-tier-audit/) for
judgment-based checks that require codebase analysis (feature coverage,
docs quality, policy evaluation).

Usage:
  npx tsx src/index.ts tier-check --repo modelcontextprotocol/typescript-sdk
  npx tsx src/index.ts tier-check --repo ... --conformance-server-cmd '...' \
    --conformance-server-cwd ... --conformance-server-url ... --output json

* refactor: revise tier-check CLI and skill based on review feedback

- Move skill to .claude/skills/ so it's auto-available in Claude Code
- Remove feature-coverage subagent (redundant with conformance tests)
- Remove hardcoded ~/src/mcp paths from all skill files
- Trim conformance server table to TS + Python only
- Rename file_existence check to policy_signals (informational, not blocking)
- Add GitHub native issue types detection to labels check
- Add missing features to docs-coverage checklist (tasks, elicitation URL mode, JSON Schema 2020-12)
- Add README with CLI quick start and escape hatch for non-Claude-Code users
- Use --limit 500 instead of --limit 100 for gh issue list

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address review feedback on skill and README

- Use npx @modelcontextprotocol/conformance instead of node dist/index.js
- Add full GitHub auth instructions (gh auth login, GITHUB_TOKEN, --token)
- Point TS SDK conformance server to typescript-sdk/test/conformance/
- Fix Python SDK URL to localhost:3001/mcp (not TBD)
- Remove manual gh issue list / gh release list from SKILL.md (CLI handles it)
- Remove Claude Code-specific subagent_type references
- Assume user is already in conformance repo
- Clean up policy-evaluation-prompt.md: remove redundant grep commands,
  focus on content evaluation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: resolve lint errors in conformance.ts

Remove unused variable assignments flagged by eslint.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style: apply prettier formatting

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: add npm run tier-check script, update docs with examples

- Add "tier-check" npm script so users can run `npm run tier-check --`
  instead of `node dist/index.js tier-check`
- Update SKILL.md, README, and skill README to use npm run tier-check
- Add full conformance examples with --conformance-server-cmd/cwd/url
  flags and realistic paths (~/src/mcp/typescript-sdk)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style: fix prettier formatting in SKILL.md

* fix: add per-scenario timeout, support url-only conformance, fix stdout pollution

- Add 30s per-scenario timeout to prevent tier-check from hanging
- Allow --conformance-server-url without --conformance-server-cmd (server already running)
- Move runner status logs to stderr so --output json produces clean JSON
- Update SKILL.md and README with --silent flag and pre-start server workflow

* refactor: skill takes local path + server URL instead of repo name

The skill now requires two arguments:
1. Local path to the SDK checkout (for direct file inspection)
2. URL where the everything server is already running

The GitHub owner/repo is derived from git remote. This eliminates
cloning, server startup complexity, and branch confusion (v1.x vs main).

* refactor: write detailed reports to files, show concise summary

Reports go to results/tier-audits/<sdk>-<date>/ (already gitignored).
Claude's console output is now just the tier classification,
pass/fail summary line, top 3 actions, and file paths.

* docs: update READMEs for new skill interface and pre-start workflow

* simplify: flat file output instead of nested directory

* fix: remediation always shows path to Tier 2 and Tier 1

* feat: add client conformance testing to tier-check CLI and skill

- Add checkClientConformance() that runs core client scenarios
  (initialize, tools_call, elicitation-defaults, sse-retry, auth)
  by spawning the SDK's conformance client via --client-cmd
- Add client_conformance to TierScorecard type
- Wire --client-cmd option into the CLI
- Update tier logic: both server + client conformance feed into
  Tier 1 (100%) and Tier 2 (>=80%) requirements
- Update terminal and markdown output to show both conformance types
- Update skill to auto-detect conformance client or accept explicit
  client-cmd argument
- Update README with new option and examples

* improve: table summary output, write reports via subagents

- Change executive summary from pipe-delimited line to a readable
  table with T2/T1 columns
- Move assessment and remediation file writing into parallel
  subagents to keep the main conversation thread clean

* improve: list tier gaps as numbered items instead of one-line blob

* improve: finalize summary format with separator, high-priority fixes, numbered gaps

* improve: add pre-flight checks for gh auth and server reachability

Fail fast with clear error messages if GitHub CLI is not authenticated
or if the conformance server URL is not reachable, rather than failing
deep into the scorecard run.

* docs: improve README and fix skill auto-detection paths

- Claude Code section: explain client-cmd auto-detection for TS/Python,
  show explicit 3-arg form for other SDKs, add examples for all three
- Fix TypeScript build command (npm run build, not pnpm build:all)
- Fix Python server command (add --port, use uv sync --package)
- Fix Python client path (.github/actions/conformance/client.py)
- Expand 'Other SDKs' section with guidance on everything server
- Add gh auth login prerequisite to Claude Code steps

* simplify: remove client-cmd auto-detection, require explicit argument

Client command is now always passed as the third argument. If omitted,
client conformance is skipped and noted as a gap. No more magic path
detection — clearer and more predictable.

* fix: align docs table with canonical list (48 features), simplify policy eval

Docs coverage:
- Table now has numbered rows matching all 48 non-experimental features
  from the canonical list (was missing 7: tools text/image/audio/embedded/
  error/notifications, protocol version negotiation)
- Hardcode total as 48 in summary so agents don't miscount

Policy evaluation:
- Simplified from deep content analysis to file-existence checks
- Dependency policy: DEPENDENCY_POLICY.md, dependabot.yml, or CONTRIBUTING.md section
- Roadmap: ROADMAP.md must exist (GitHub milestones alone not sufficient)
- Versioning: VERSIONING.md or CONTRIBUTING.md section
- Removed GitHub API calls for milestones and releases from policy eval

* refactor: extract canonical feature list into single source of truth

Create references/feature-list.md with all 48 non-experimental + 5
experimental features. The docs-coverage prompt now references this
file instead of duplicating the list. One place to update when
features change.

* fix: separate deterministic file checks from AI content evaluation

CLI (files.ts): now checks all policy files deterministically —
DEPENDENCY_POLICY.md, docs/dependency-policy.md, dependabot.yml,
renovate.json, ROADMAP.md, docs/roadmap.md, VERSIONING.md,
docs/versioning.md, BREAKING_CHANGES.md (in addition to existing
CHANGELOG.md, SECURITY.md, CONTRIBUTING.md).

AI policy eval: receives CLI output showing which files exist,
then reads ONLY those files to judge content quality. No longer
searches the repo for files — clean separation of concerns.

* style: apply prettier formatting

* revert: undo unrelated console.log change in runner/server.ts

* refactor: shell out to conformance CLI instead of reimplementing runner

Address PR feedback: conformance.ts was duplicating the normal conformance
running code. Now shells out to 'node dist/index.js server/client' with
-o to save results to a temp dir, then parses the checks.json files.

Also removes --conformance-server-cmd and --conformance-server-cwd options
since the server must be pre-started.

* docs: add Go and C# SDK examples to README and SKILL.md

* fix: add --framework net9.0 to C# server command

* rename: conformance.ts -> test-conformance-results.ts

Avoids confusion with src/runner/ (the actual conformance runner).
This file just invokes the CLI and parses output.

* style: prettier formatting

* fix: reconcile conformance results against full scenario list

The tier-check CLI was only counting scenarios that produced a
checks.json file. Scenarios that crashed or failed to run (e.g., auth
scenarios when OAuth is not implemented) were invisible, making the
denominator artificially small (e.g., 4/4 instead of 4/23).

Now both checkConformance and checkClientConformance reconcile their
parsed results against the known scenario lists, adding failure entries
for any expected scenario that didn't produce results.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: tighten documentation evaluation criteria

Clarify what counts as documented vs just having code:
- Conformance test servers don't count as docs or examples
- Examples without prose = PARTIAL, not PASS
- Go Example* test functions explicitly allowed
- Clear PASS/PARTIAL/FAIL verdict definitions

* docs: add Labels and Spec Tracking rows to audit report templates

The executive summary and assessment report were missing two SEP-1730
requirements: label taxonomy compliance and spec tracking (new protocol
features timeline).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: reuse ConformanceCheck type from src/types.ts instead of redefining

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…elcontextprotocol#148)

parseOutputDir only read one level deep, so scenarios with '/' in their
name (e.g. auth/metadata-default) were stored in nested subdirectories
and never found. This caused tier-check to report 4/23 client conformance
when the actual pass rate was 23/23.

Also removes accidentally committed pnpm-lock.yaml.
* feat: add conformance tests for SEP-990

* Resolving review changes: Removed redundant tests, updated audience params

* fix: unused serverUrl parameter in runCrossAppAccessTokenExchange

* chore: apply prettier formatting

* fix: address PR review comments for SEP-990 conformance tests

- Delete unused separate token-exchange and jwt-bearer scenarios,
  keeping only the complete e2e flow (review comment)
- Add missing required token exchange params per SEP-990 spec:
  requested_token_type, audience, resource (review comment)
- Use ctx.idp_client_id for token exchange client_id instead of
  AS client_id (review comment)
- Client discovers resource and auth server via PRM metadata
  instead of receiving auth_server_url via context (review comment)
- Server IdP handler verifies all required token exchange params
  with detailed error messages (review comment)
- Add resource, client_id, jti claims to ID-JAG per SEP-990 spec
- Verify ID-JAG typ header (oauth-id-jag+jwt) in JWT bearer handler
- Remove auth_server_url from context schema

* feat: add client auth and ID-JAG validation to XAA conformance test

Server-side (AS) now verifies:
- client_secret_basic authentication on JWT bearer grant
- ID-JAG typ header is oauth-id-jag+jwt
- ID-JAG client_id claim matches the authenticating client (Section 5.1)
- ID-JAG resource claim matches the MCP server resource identifier
- Client credentials provided via context (client_secret)

Server-side (IdP) now:
- Sets ID-JAG client_id to the MCP Client's AS client_id (not the
  IdP client_id), per Section 6.1

Example client now:
- Authenticates to AS via client_secret_basic (Authorization: Basic)
  instead of sending client_id in body
- Checks AS metadata grant_types_supported includes jwt-bearer
  before attempting the flow

* fix: share MockTokenVerifier and remove unadvertised auth method

- Add shared MockTokenVerifier between AS and MCP server so the MCP
  server only accepts tokens actually issued by the auth server,
  matching the pattern used by all other auth scenarios
- Remove private_key_jwt from tokenEndpointAuthMethodsSupported since
  the handler only implements client_secret_basic

---------

Co-authored-by: Paul Carleton <paulc@anthropic.com>
…ier (modelcontextprotocol#138)

The SDK's `requireBearerAuth` middleware only converts `InvalidTokenError`
instances to HTTP 401 responses. Generic `Error` instances fall through
as HTTP 500, which prevents clients from detecting authentication failures
and initiating the OAuth refresh/re-auth flow.

This was discovered while building token refresh conformance scenarios —
the mock server was returning 500 for expired/invalid tokens instead of
the expected 401.

Co-authored-by: JD Maturen <70791+jdmaturen@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
…ontextprotocol#147)

* feat: add specVersions classification to conformance scenarios

Each scenario declares which spec versions it applies to as a list.
Scenarios that carry forward (e.g. initialize) list all applicable versions
['2025-06-18', '2025-11-25']. Scenarios removed from newer specs (e.g.
backcompat auth) only list their original version ['2025-03-26'].

- specVersions list on Scenario and ClientScenario interfaces
- --spec-version CLI filter uses simple .includes()
- Tier-check conformance matrix (Server / Client: Core / Client: Auth)
  with per-version columns and unique All* count
- 7 unit tests for specVersions helpers
- Updated tier-audit skill docs with matrix format

* fix: fix console output template for tier audit skill

The template had an orphan table header (Check | Value | T2 | T1)
with no rows above the conformance matrix, causing an empty table
to render. The scorecard rows below the matrix also lacked their
own header.

Fix: two self-contained tables with clear labels — 'Conformance:'
for the per-version matrix, 'Scorecard:' for the check rows.

* fix: align skill console template with tier-check script output

Add asterisk footnote ('unique scenarios — a scenario may apply to
multiple spec versions') and rename 'Scorecard' to 'Repository Health'
to match the labels used by the tier-check CLI output.

* fix: add specVersions to CrossAppAccessCompleteFlowScenario

Added in 83c446d on main after this branch diverged.
…extprotocol#153)

* fix: pass --branch to tier-check CLI in tier-audit skill

The skill was not forwarding the --branch argument to the tier-check
CLI, causing policy signal checks to always run against the repo's
default branch. Files on feature branches showed as 'Not found.'

Now derives the branch from the local checkout if not explicitly
provided, and always passes it to the CLI.

* fix: exclude draft/extension scenarios from tier-scoring conformance rates

SEP-1730 says date-versioned scenarios count toward tier scoring while
draft and extension scenarios are informational. The CLI was including
all scenarios in pass_rate, causing extension-only failures to block
Tier 1.

Changes:
- pass_rate now only counts scenarios with at least one date-versioned
  spec version
- Terminal and markdown output split into a tier-scoring matrix
  (date-versioned + All*) and an informational section (draft/extension)
  that only renders when there are draft/extension scenarios
Ensures the `resource` field is present and matches the server URL
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants